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ABSTRACT 

The paper outlines the concepts and main elements of a RObot Technology Experi- 
ment (ROTEX) proposed by DLR to fly with the next German spacelab mission, D2, in 
December 1991. It provides a 1-meter size, six-axis robot inside a spacelab rack, equipped 
with a multisensory gripper (force-torque sensors, an array of range finders, and mini stereo 
cameras). The robot will perform “assembly” and “servicing” tasks in a generic way, and 
will grasp a floating object. The paper focusses on the man-machine and supervisory control 
concepts for teleoperation from the spacelab and from ground, and explains the predictive 
estimation schemes for an extensive use of tune-delay compensating 3D computer graphics. 

A JPL-NASA proposal is underway to join ROTEX with a TeleRobotic Intelligent In- 
terface Flight Experiment (TRIIFEX), utilizing the functional and operational capabilities 
of ROTEX. The main objective of TRIIFEX is to extend performance and operation experi- 
ence with hybrid position and force-reflecting control of telemanipulators to space telerobot 
missions, and to evaluate its human factors implications. JPL is planning to build a general- 
purpose computerized force-reflecting position control device backdriveable from robot hand 
sensors and a complementary graphics display of robot hand sensor data. The paper will 
include a brief description of the main elements of TRIIFEX, their interfaces to ROTEX, 
and the specific TRIIFEX objectives. TRIIFEX operation is planned from onboard the 
spacelab and from ground. 

INTRODUCTION 

Among the many areas important in space technology, automation and robotics (A&R) 
will become one of the most attractive ones for smaller countries like the Federal Republic 
of Germany, as well as for the big space nations. It will allow experiment-handling, material 
processing, assembly and servicing with a limited amount of manned missions, and it will 
provide an extensive technology transfer from space to earth applications. This is one of 
the main reasons why several activities towards space robotics have started in Germany 
with the long-term goal to make a major contribution to the space station, e.g., to the Man 
Tended Free Flyer (MTFF) subsystem. 
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In addition to study activities, DLR (the German Aerospace Research Establishment) 
made a proposal at the end of 1985 to fly a robot technology experiment ROTEX with the 
next “German” spacelab mission, D2, scheduled now for December 1991. Phase C/D is run- 
ning now, with participation of two major German space technology companies, DORNIER 
and MBB, and including several of the leading German robotic research institutes. Thus 
ROTEX is a starting shot for a German participation in space automation and robotics, 
with a broad national basis. 

A JPL-NASA proposal is underway to join ROTEX with TRIIFEX, utilizing the func- 
tional and operational capabilities of ROTEX. TRIIFEX employs hybrid position and force- 
reflecting master-slave control for telemanipulation. This control technique is the most ef- 
ficient one for versatile telemanipulation in terrestrial applications; this control is standard 
in the nuclear industry. The reason for the efficiency of this control is twofold: (i) direct 
position control is inherent to these systems, and (ii) the operator’s hand receives a genuine 
impression of acting forces and thereby is dynamically connected to the remote control task. 
However, system performance in this mode of control is closely coupled to the operator’s 
body (manual) and mental (model reference) performance capabilities. The basic question 
TRIIFEX is asking is: how can this control technique be extended to space efficiently and 
safely? In particular: (i) How does microgravity affect on-board operator’s performance? 
(ii) How will ground operator relate to control actions in micro-g from control inputs in 
normal-g, in particular, in the presence of a several-second R/T communication time delay? 
The first specific question is related to the operator’s neuromotor response characteristics in 
micro-g. The second question is related to the operator’s psychomotor response character- 
istics when control actions are across basically different dynamic environments and across 
time delay. 

The first part of the paper is devoted to the description of ROTEX, and the second 
part briefly summarizes the main elements of TRIIFEX and interfaces to ROTEX. 

THE ROTEX PROJECT 

The ROTEX system contains several items: 

• A small, six-axis robot (work space lm) inside a space-lab rack (Fig. 1). The robot 
arm will be built by DORNIER company. Its gripper, built by DLR, will be provided 
with a number of sensors (Fig. 2): two six-axis force-torque wrist sensors, a tactile 
array in each finger for grasping force control, an array of nine laser-range finders, and 
a tiny stereo camera (smaller than a match-box) to provide a stereo image out of the 
gripper. In addition, a fixed pair of cameras will provide a stereo image of the robot’s 
working area. 

• The robot is able to perform automatic, preprogrammed motions as well as teleoperated 
motions via an astronaut on board or an operator on ground (Fig. 2). 

• Two types of operational modes will be performed by the robot: 

a) Experiment handling. This is a slow or “micro-gravity (/jg) mode” based on the 
execution of preprogrammed paths that may be reprogrammed from ground. 
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b) Servicing. This is a fast mode based on teleoperation on board and from ground, 
and on sensor-based learning of tasks on ground which are executed automatically 
on board. 

• The main goals of the experiment are 

a) To verify joint control (including friction models) under zero gravity, as well as fig 
motion planning concepts, based on the requirement that the robot’s accelerations 
while moving must not disturb any fig experiments nearby. 

b) To demonstrate and verify the use of advanced six dof hand controllers under zero 
gravity. 

c) To demonstrate the combination of a complex, multisensory robot system with 
powerful man-machine interfaces (such as 3D computer graphics, control balls, 
force-reflecting hand-controllers, stereo imaging, voice input-output) that also al- 
low for teleoperation from ground. 

In order to demonstrate servicing capabilities by teleoperation, three basic tasks are 
envisioned: 

a) Assembling a mechanical grid structure (Fig. 3). 

b) Connecting/disconnecting an electrical plug (which stands for replacement of an ORU). 

c) Grasping a floating object. 

For all these tasks, continuous or on-line sensory feedback is involved. 

Multisensory Robot Gripper 

Multiple sensing in the robot gripper and sensory feedforward in the man-machine 
interface are the key for the telepresence concepts envisioned. The gripper sensors involved 
belong to the new generation of DLR robot sensors with all analog preprocessing and digital 
computations performed inside the sensors or at least in the robot’s wrist (Fig. 4). Using 
a high-speed serial bus, only two signal wires come out of the gripper (carrying signals of 
forces-torques and distances), augmented by two 20 kHz power supply wires from which 
the sensors themselves derive their DC power supply voltages via tiny transformers. The 
following sensor modules are provided: 

a) An array of nine laser range finders based on triangulation: one “big” sensor (half the 
size of a match box) switchable into a scanning mode for a longer range of « 3-50 cm, 
and four smaller ones in each finger for shorter ranges of 0-3 cm. The range finders are 
the result of more than five years’ development aiming at a precise performance over a 
remarkable range and independent of the slant angle and surface of the measured object. 
One of the main problems to be solved in this development was the design of a nonlinear 
digital control system that adapts the light transmitter’s intensity depending on the 
reflected light intensity. The signal control system now used varies the emitted power 
in a range of 1 to 10,000 within 10 /xsec. This indeed enables the sensors to measure 
distances with respect to surfaces that show up strongly with quickly changing reflection 
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characteristics. 

b) A “stiff” six-axis force-torque sensor based on strain-gauge measurements and a “com- 
pliant” optical sensor (Fig. 5). Originally, it seemed necessary to make a selection 
between these two sensing principles. A solution was found that combines both princi- 
ples in one compact sensor with the option to switch between them during operation. 
The “compliant” optical force-torque sensor consists of an inner and an outer part (Fig. 
5). The basic measuring arrangement in the inner ring is composed of an LED, a slit 
and, perpendicular to it, a linear position sensitive detector (PSD) which is mobile 
against the remaining system. Six such arrangements (rotated by 60 degrees each) 
are mounted in a plane, whereby the slits alternately are vertical and parallel to the 
plane. The ring with PSDs is fixed inside the outer part and connected by springs to 
the LED slit basis. The springs bring the outer part back to neutral position when no 
forces/ torques are exerted. 

c) A tactile array of four by eight sensing dots in each finger using elastomeric rubber as 
transducer. 

d) A pair of tiny stereo cameras, augmented with an additional pair of stereo cameras 
which is fixed in the rack, yielding a global view of the work space. 

e) The sensor or control ball as a six dof hand controller. For a very natural six degree-of- 
freedom control of robots and of 3D computer graphic objects by using only one human 
hand, DLR developed different types of plastic hollow balls with six-axis force-torque 
sensors inside [3,4]. The latest and preferred version uses the compliant sensor (Fig. 
5) inside the ball. The only difference between the wrist sensor and the control ball is 
that the outer ring in Fig. 5 is replaced by a plastic hollow ball. 

Sensory Feedback Structures 

The use of sensors in the feedback control is based on a sensor-based fine motion plan- 
ning concept that has been outlined in different papers (e.g. [7]). Its main features are briefly 
as follows (see also Fig. 6). “Rudimentary” commands are derived either on-line from a hu- 
man teacher operating the control ball or from a path-generator connecting preprogrammed 
points. They are interpreted in a dual way as force/torque or positional/orientational com- 
mands. When the robot moves in free space, the ball forces are transformed into trans- 
lational commands; when the robot senses contact with the environment, it takes the ball 
inputs as nominal force values and, by closing the sensory loop at the robot’s site (see Fig. 
2), it always exerts only those forces which are given by the human operator [8]. Of course, 
any kind of shared control between robot and operator is feasible. Though the forces are 
not fed back to the human arm (as in “bilateral” force control), the operator is sure that 
the robot is fully under his control and he easily may lock up doors, assemble parts or plug 
in connectors. In other words, the human operator (via stereovision and 3D graphics) is 
enclosed in the feedback loop on a high level but with low bandwidth, while the low-level 
sensory loops are closed on-board at the robot directly with high bandwidth. Thus a su- 
pervisory control technique is envisioned that permits shifting more and more autonomy to 
the robot while always offering real-time human interference (Fig. 7). 
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Visual Feedback and Predictive Control 

In teleoperation on-board the spacelab, the visual display is restricted to the use of a 
small colour TV monitor. In the present state of planning, the B/W stereo images produced 
by the gripper or the global cameras are displayed alternately to the operator, who will use 
shutter glasses (developed in German nuclear power facilities with only 15V power supply 
and switching frequencies up to 1 kHz) to obtain stereo perception of images. The sensory 
information will be added in simple bar-like form at the monitor’s edges. 

For teleoperation from ground the situation is different: much more powerful equipment 
is available there for visual feedback, but the communication link restrictions are obvious. 
Indeed, it turned out that the “normal” spacelab up-links as used until now are not at 
all adequate for telepresence ideas. They would create up-link delays of up to 15 seconds, 
partly caused by data checks in Houston. This seemed to destroy the ground teleoperation 
concept completely. The present base-line uses the Text And Graphics channel (TAG) for 
the up-link, eliminating these difficulties. This channel uses the TDRS satellite, and could 
not be tested until now. Using the TAG channel, the up-link command rates are in the 
range of 2 kbit/sec, assuming a sampling rate of 20 Hz. Nevertheless, we have to take into 
account an overall delay of four seconds in the loop closed at the ground station. In order 
to get exact knowledge about this delay, we will provide the ball commands with a code 
which, when arriving at the robot, are packed into the down-link information. 

The down-link information comprises a sequential RGB video signal. The left and 
right black-and-white stereo images are packed into the red and green channel. They are 
superimposed and displayed on a polarized screen on ground. The down-link data channel 
also contains all internal (position encoders) and external sensory signals so that on a 3D 
graphics monitor the robot’s position is displayed as well as all sensor data. Preferably, a 
stereo graphics system is used with real-time volume-shaded representation of the workcell. 

The big problem for teleoperation from ground is the communication time delay. The 
only way to compensate for it is by using predictive computer graphics. Extensive use of 
them will be made in ROTEX. Fig. 8 shows that the human operator at the remote work- 
station handles the six dof hand controller by looking at a “predicted” graphics (e.g., wire 
frame) model of the robot. The control commands issued to this instantaneously reacting 
robot simulator are sent to the remote robot as well, using the time-delayed transmission 
links. Now the ground-station computer and simulation system contains a model of the 
up-link and down-link delay lines as well as a model of the actual states of the real robot 
and its environment. Note that we have several alternatives to superimpose the predicted 
robot model (augmented by predictions of any other moving parts) with other information 
representations: 

a) The presently received (of course, delayed) TV stereo or mono image in case the globally 
fixed camera pair is active. 

b) The “delayed” graphics image derived from this delayed TV image (including the case 
of hand-mounted cameras) and other sensory data. 
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c) The actual graphics image as derived from the state space model of robot and environ- 
ment. 

There is not yet a final conclusion on what the most efficient method of superposition 
would be. There is, of course, evidence that the most crucial problems lie in the derivation of 
“output data” (e.g., positions/orientations of moving objects) from stereo images and range 
finders. As real-time is required, this is an extremely challenging preprocessing problem 
solved by a parallel transputer system but not discussed in more detail here. 

For the robot, we assume a linearized Cartesian state space model x* +1 = Ax k + bu h . 
In the case of grasping a floating object, this model in standard form of digital control theory 
not only describes the Cartesian robot dynamics, but also the dynamics of the free-flying 
part. 

Thus the left part of Fig. 8 is just a prediction of the robot’s present estimated state 
Xk to the future state n u is the up-link delay time expressed as a multiple of the 

sampling period, that makes up one delay d. This predicted state is the state to which the 
presently issued hand controller command has to refer. But the more interesting part is 
the estimator on the right half of Fig. 8. It compares the measured, but down-link-delayed 
output data V_ k _ ni (the robot’s positions and orientations) to the output data from 

the robot model running through the down-link-delay computer model (n<* is the number 
of sampling periods in the down-link delay). The estimator’s detailed structure has been 
derived in [9]. For telemanipulation from ground in case of an assembly operation and for 
sensor-based task learning on ground, a realistic graphic simulation of the workcell and the 
robot’s sensory perception is the crucial item. Fig. 9 shows the envisioned structure for 
telemanipulation from ground with simulated sensory path refinement. 

TRIIFEX PROJECT 

The key element in the TRIIFEX project is the use of a Force Reflecting Hand Controller 
(FRHC) to control the robot both from an on-board control stand and from a ground 
control station. The planned FRHC is not a geometric replica of the robot arm; it is a 
generalized position input and force feedback device, tailored to the operator and to the 
control station, and applicable to different manipulators. The generalized FRHC technique 
has been described in detail elsewhere [11], The device planned for TRIIFEX is somewhat 
different from the one described in [11] for packaging reasons; it will have an elbow instead 
of a telescoping linear link. 

The use of a generalized (non-replica) FRHC device also represents a new control 
configuration: (i) force feedback is referenced to wrist force-torque sensor information, and 
(ii) the control requires a computer for coordinate transformations and sensor data handling. 
The sensor data will be displayed on a dedicated graphics display. The planned on-board 
TRIIFEX system is shown schematically in Fig. 10. This figure emphasizes the on-board 
TRIIFEX electronics architecture and its interface to the ROTEX electronics. 

The performance capabilities and characteristics of a generalized FRHC laboratory 
system at JPL are described elsewhere in this conference proceedings [12-13]. The on-board 
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experiments are planned to be identical to the ROTEX experiments. 

The TRIIFEX ground station system is schematically shown in Fig. 11. A key compo- 
nent in the TRIIFEX ground system is the use of a “Phantom Robot,” which is a high-fidelity 
3D graphics image of the real robot, superimposed on the 3D TV image of the real robot 
in the workcell. The operator interacts with the Phantom Robot in real time. Thus, the 
motion of the Phantom Robot on the TV monitor screen acts as a predictive display in a 
real work environment shown on the TV screen. The motion of the real robot image will 
follow the motion of the Phantom Robot image after some time delay. The contact closure 
actions will be referenced to local F /T sensor data and will be controlled locally through the 
F/T sensor data upon the operator’s initialization commands. The operator’s responsibility 
here is the verification of the status of the real robot versus the Phantom Robot before the 
closure action is initiated so that there is a certainty that the local control algorithm can 
complete the task. Again, the TRIIFEX ground experiments are planned to be identical to 
the ROTEX experiments. 

The general objective of the TRIIFEX project is to validate and quantify force-reflecting 
position control technology for Earth-orbital space missions. The planned performance mea- 
surements are focussed on human operator’s performance capabilities. They are aimed to 
evaluate (i) on-board operator’s ability to use force-reflecting position control of a tele- 
manipulator in microgravity, and (ii) ground operator’s ability to use this technique for 
telemanipulation in microgravity from a normal gravity base under several-second R/T 
communication time delay. 

An expected major benefit of the TRIIFEX project is the evaluation of the validity of 
ground simulation data of microgravity telemanipulation by comparing flight experiment 
data to data obtained through ground simulation of the same experiments. 

CONCLUSION 

The ROTEX proposal is a first step of Germany’s engagement in space robotics aimed 
at the demonstration of a fairly complex system with a multisensory robot on board and 
human telerobotic interference that makes use of sensor-based six dof hand controllers, new 
concepts for predictive 3D computer graphic and stereo display. Teleoperation from ground 
is a very challenging technique that forces us to move even more strongly toward on-board 
autonomy. The planned control strategy is to move the human operator increasingly towards 
supervisory control without changing the control loop structures. 

The TRIIFEX proposal complements the ROTEX proposal by providing alternative 
man-machine interface devices and techniques in order to broaden the knowledge base for 
human-control performance capabilities for space telemanipulation. 
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Fig. 2 Schematic Representation of ROTEX 
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